1. Technical Field
This invention generally relates to the testing of integrated circuits, and more specifically relates to a computer mechanism and method for testing an integrated circuit for compliance with its architectural design parameters.
2. Background Art
The proliferation of modern electronics into our everyday life is due in large part to the existence, functionality and relatively low cost of advanced integrated circuits. As technology moves ahead, the sophistication of integrated circuits increases. An important aspect of designing an advanced integrated circuit is the ability to thoroughly test the design of the integrated circuit to assure the design complies with desired architectural, performance and design parameters. Testing a complex integrated circuit such as a super scaler microprocessor requires the generation of a large number of instruction sequences to assure that the microprocessor behaves properly under a wide variety of circumstances.
Referring to FIG. 2, one known system 200 for testing the design of a complex integrated circuit represents a method developed by IBM for integrated circuit design verification. Note that the term "integrated circuit" used in this specification refers to a single integrated circuit or a collection of integrated circuits that work to perform desired functions. System 200 includes a representation 210 of the integrated circuit in a hardware description language, such as VHDL or Verilog. A hardware description language is a computer-readable language that defines functional and performance parameters for the integrated circuit. The hardware description language representation 210 is compiled in step 220, which yields a simulation model 227. The simulation model 227 is a representation of all the components and their interconnections on the integrated circuit.
Simulation model 227 is used by a gate level cycle simulator 228 to perform test cycles to test the integrated circuit design. In addition, gate level cycle simulator 228 uses data from one or more testcases 225 to perform the cycle-by-cycle testing of the integrated circuit design. Testcases 225 may be generated by a testcase generator 224, which generates the testcases 225 in accordance with parameters specified in a testcase definition file 223. If testcase definition file 223 does not specify any parameters, testcase generator 224 generates truly random testcases. If, however, the testcase definition file 223 specifies one or more parameters, these parameters provide biasing to the testcase generator 224, which causes testcase generator 224 not to generate truly random testcases, but to generate testcases that are biased according to the parameters specified in testcase definition file 223. Testcase definition file 223 therefore provides a mechanism for biasing or "steering" the testcase generator to generate testcases that are more likely to test certain aspects of the integrated circuit design. An alternative to automatically generating somewhat random test cases using testcase generator 224 is to provide manually-written testcases 222 that are written by a designer to test the integrated circuit design for specific behavior.
In addition to the representation of the integrated circuit in hardware description language 210, there is also an architectural model 230 that defines the high-level architecture of the integrated circuit. This architectural model 230 specifies elements and features of the integrated circuit at a relatively high level. For example, the architectural model 230 for a microprocessor would include a specification of the number of general-purpose registers, the size of memory, the configuration of the program counter, etc. A simulator 240 uses the testcases 225 to generate expected results 260 that correspond to each test case. In addition, testcase generator 224 uses information from architectural model 230 to generate appropriate testcases 225 to test the integrated circuit design.
Testcases 225 may also be grouped into certain sets or subsets of tests to provide a greater likelihood of fully testing the integrated circuit design. Regression bucket 221 in FIG. 2 represents a container for groups of tests known as regression suites. The concept of regression suites is well-known in the art.
Gate level cycle simulator 228 uses testcases 225 to perform cycle-by-cycle tests of the integrated circuit design, typically using a single testcase for each simulation. When a testcase has been simulated by gate level cycle simulator 228, the results of the simulation are compared to the expected results 260 that correspond to the testcase that was just simulated. If the simulation results match the expected results 260 for the testcase, the testcase "passes", and the results of the simulation are used in determining test coverage of the integrated circuit design. If the simulation results do not match the expected results 260 for the testcase, the testcase "fails", and the results of the simulation are not used in determining test coverage, but rather the results of the failing test are examined to determine what failed in the integrated circuit design. When a testcase fails, the reason for the failure is repaired in the design, and the testcase is run again.
Once gate level cycle simulator 228 has performed test cycles that use all the testcases 225, the human operator running the tests typically evaluates test results 250 to determine how completely the gate level cycle simulator 228 has tested the design of the integrated circuit. Known methods of evaluating test results 250 to determine test coverage are very simplistic. The term "test coverage" as used in this specification relates to how completely the test patterns have tested the integrated circuit design.
One known way to attempt to get good test coverage is to run a set of one or more Architectural Verification Programs (AVPs) that test each use of every instruction in the architecture. However, running AVPs gives no information regarding how each test pattern actually runs on the integrated circuit. Again, using the example of a super scaler microprocessor, running AVPs gives no indication of whether instructions run in a non-pipelined manner or whether they run in a super scaler (i.e., pipelined) manner.
Another simplistic known way to evaluate test pattern coverage checks to see if all the pertinent signal lines in the integrated circuit change state during simulation of all the test patterns. Yet another simplistic known way to evaluate test pattern coverage looks for a particular event or sequence of events while running the test patterns, or looks to see if the gate level cycle simulator 228 passes through a particular state. These known methods of evaluating test coverage for an integrated circuit are woefully inadequate for complex integrated circuits such as super scaler microprocessors.
In sum, using prior art techniques of evaluating test pattern coverage, the human operator, by either manual inspection of test results or by using computers or other equipment to measure the quality of the test results, determines whether the integrated circuit has been fully tested, or whether the testcases have not fully tested all pertinent aspects of the design of the integrated circuit. For a relatively complex integrated circuit such as a super scaler microprocessor, it is common that not all of the desired functions are tested by gate level cycle simulator using the somewhat randomly-generated testcases. Furthermore, the very simplistic methods of evaluating test coverage that are known in the art provide very limited information regarding test coverage. As a result, certain bugs in the design of the integrated circuit may go undetected. Without an improved mechanism and method for testing an integrated circuit, the design of complex integrated circuits will not be fully tested, resulting in bugs in the design that are difficult to track down and very expensive to fix.